Local positioning apparatus, and a method therefor

ABSTRACT

An apparatus and method for correctly determining the position of a vehicle in a traffic lane by obtaining correct information about the position of the traffic lane without being affected by variations in the road surface, weather, time of day, or such imaging conditions as fixed or moving lighting, are provided. An edge signal of a high spatial frequency component and a luminance signal of a low spatial frequency component of a digital image signal representing the view of the local area to the front of a vehicle are extracted. A road contour signal is then extracted from the edge signal, and a road region signal is extracted from the luminance signal. The position of the lane Sre is then detected with high precision by evaluating the lane contour Sre based on the road region signal Srr and lane contour data Sre.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to a local positioning apparatus used in alocal positioning system particularly suited to detecting the conditionof a subject, such as an automobile or other motor vehicle, on a roadwhile either stationary or moving based on such local positioninginformation as the relative location, velocity, and attitude of thesubject within a localized region with reference to an image obtained byimaging the area in front of the subject. More specifically, the presentinvention relates to a local positioning apparatus for correctlydetecting the position of a subject within a localized region withoutbeing affected by changes in the condition of the road surface, weather,time of day, fixed lighting, moving lighting, and other changes in theimaging condition.

A local positioning apparatus according to the prior art used in anautomobile is shown in FIG. 25 and described below. As shown in FIG. 25,this conventional local positioning apparatus LPP comprises an edgeextractor 1P, threshold generator 3P, contour extractor 5P, matchingoperator 9P, lane (marker) contour extractor lip, region limiter 13P,current position detector 15P, curvature detector 17P, and yaw angledetector 19P.

The edge extractor 1P is connected to a digital imaging apparatus 100(FIG. 1). The digital imaging apparatus 100 is mounted to the subject,which in this explanation of the prior art and the specification of thepresent invention below is, by way of example only, an automobile Am(FIG. 4), for capturing a perspective image Vi of the view to the frontof the automobile AM in the direction of travel, and generating adigital image signal Si of the perspective image Vi. Included in theperspective image Vi are an image of the road, the lane markers Lm1 andLm2 defining the boundaries (sides) of the lane Lm in which theautomobile AM is currently travelling, and a lane marker Lm3 definingthe far boundary of an adjacent lane (FIG. 5). The edge extractor 1Pextracts the edge pixels of the lane markers Lm1, Lm2, and Lm3 from thedigital image signal Si, and generates an extracted edge pixel signalSx'. The extracted edge pixel signal Sx' contains only the edge pixelsextracted by the edge extractor 1P, and thus represents an extractededge image Vx'.

Using a known method, the threshold generator 3P scans the extractededge pixel signal Sx' to extract a line for each of the lanes Lmdelineated by the lane markers Lm1, Lm2, and L3, and determines athreshold value Eth' for extracting pixels representing the contours ofthe lane markings from the extracted edge pixel signal Sx'. Using thesupplied threshold value Eth', the contour extractor 5P then extractscontour pixels and generates an extracted contour signal Sc'representing the contour lines of the lane markings.

The matching operator 9P then determines the line segment or arcmatching the contour lines contained in the extracted contour signalSc', and generates matching data Sm' containing all of the matching lineand arc segments.

The lane contour extractor 11P then compares the matching data Sm' withtypical lane dimension characteristics stored in memory to extract thematching line elements meeting these dimensional criteria as the contourlines of the lane, and outputs the result as lane extraction signalSmc'.

Based on this lane extraction signal Smc', the region limiter 13Pdefines a certain region around the extracted lane, and generates aregion signal Sr' delimiting this lane region. By feeding this regionsignal Sr' back to the edge extractor 1P, the edge extractor 1P limitsthe area within the perspective mage Vi used for edge extraction to theregion limits defined by the region limiter 13P.

Using the lane extraction signal Smc' from the lane contour extractor11P, the current position detector 15P detects the position of theautomobile AM on the road, or more specifically in relationship to thelane being followed.

The curvature detector 17P detects the curvature of the lane beingfollowed while the automobile AM is moving. The yaw angle detector 19Pdetects the angle of the automobile AM relative to the lane, i.e.,whether the automobile AM is travelling parallel to the sides of thelane or is following a course while would result in the automobile AMleaving the current lane being followed.

It should be noted that all of the processes described above are basedon the perspective image Vi of the area to the front of the vehicleobtained by the digital imaging apparatus 100. It is obvious, however,that correct information about the lane dimensions cannot be obtainedfrom the perspective image Vi because the perspective image is a simpletwo-dimensional representation of three-dimensional space. Specifically,shapes change in the perspective image Vi as the distance from thedigital imaging apparatus 100 increases with the size of an object at adistance from the digital imaging apparatus 100 being displayed smallerthan the same object in close proximity to the digital imaging apparatus100. In addition, the edges of a road or lane are indistinct in aperspective image Vi, making edge detection difficult.

Road conditions are also not constant, and this further complicates roadedge detection. For example, recognizing the contour of a road or laneby means of edge detection is, in fact, impossible when the side of aroad or a lane marker is hidden by such as vegetation, dirt, or gravel.Edge detection is also not practically possible when the lane markersare not recognizable in part or in full because of soiling, damage, orother cause.

The brightness of the road surface is also extremely variable, and isaffected by such factors as the weather, whether the road is inside atunnel, and whether it is day or night. Stationary lights installed intunnels and beside the road also only partially and locally illuminatethe road surface, and spots of extreme brightness or darkness can resulteven from the same road surface. These conditions are furthercomplicated at night by irregular changes in illumination resulting bothfrom the headlights of the subject automobile AM and the headlights ofother vehicles. Accurately determining the edge detection thresholdvalue Eth' is effectively impossible under environments subject tochanges in driving conditions, time of day, and weather, as well asdynamic changes in brightness in the perspective image Vi due to fixedor moving lighting even when the driving conditions, time, and weatherremain constant.

In other words, it is not possible to obtain accurate dimensionalinformation about the road and lane from an extracted contour signal Sc'that is based on such an inaccurate threshold value Eth'.

It is therefore clearly extremely dangerous to detect the localpositioning of a vehicle relative to a road or lane based on suchunreliable, inaccurate, and-distorted dimensional information, and todetect the road curvature and vehicular yaw based on such erroneouslydetermined positioning information.

SUMMARY OF THE INVENTION

The object of the present invention is therefore to resolve the aboveproblems by providing a local positioning apparatus for detecting thelocal position of a subject capable of advancing in a direction relatedto a lane within a localized region based on a digital image signalrepresenting a localized region in the direction of subject travel. Toachieve this object, the local positioning apparatus comprises a firstimage signal generator for extracting a high frequency component of aspatial frequency from the digital image signal to generate an edgesignal; a contour extractor for extracting lane contours based on theedge signal, and generating lane contour data; a second image signalgenerator for extracting a low frequency component of a spatialfrequency from the digital image signal to generate a luminance signal;a lane area extractor for extracting the lane area based on theluminance signal, and generating lane area data; and a lane detector fordetecting a lane position and generating a lane detection signal basedon the lane contour data and lane area data.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects and features of the present invention willbecome clear from the following description taken in conjunction withthe preferred embodiments thereof with reference to the accompanyingdrawings throughout which like parts are designated by like referencenumerals, and in which:

FIG. 1 is a block diagram of a local positioning apparatus according toa first embodiment of the present invention,

FIG. 2 is a detailed block diagram of essential components of a localpositioning apparatus shown in FIG. 1,

FIG. 3 is used to describe the coordinate system of a perspective imagein the present invention,

FIG. 4 is used to describe the coordinate system of a birds'-eye imagein the present invention,

FIG. 5 is a perspective image of the view to the front of the automobileAM that is processed by the local positioning apparatus of theinvention,

FIG. 6 shows the edges extracted from the perspective image in FIG. 5,

FIG. 7 is used to described a method of extracting contours from theedge image in FIG. 5,

FIG. 8 shows the contour lines extracted from the edge image in FIG. 5,

FIG. 9 is used to describe how the processing region of the perspectiveimage is limited by the contour extraction region limiter in FIG. 2,

FIG. 10 is used to describe a perspective image after region limited bythe method shown in 9,

FIG. 11 is used to describe the luminance image extracted from the frontperspective image shown in FIG. 5,

FIG. 12 is a graph of the relationship between pixel luminance in theluminance image in FIG. 11 and the pixel count having a particularluminance value,

FIG. 13 is a graph of experimental values showing the relationshipbetween the luminance values of pixels in the road area of the luminanceimage, and the focal distance of the road surface from which thecorresponding pixel was captured,

FIG. 14 is used to describe the road brightness setting method of theroad brightness setter shown in FIG. 2,

FIG. 15 is used to describe the road area extraction method of the roadimage extractor in FIG. 2,

FIG. 16 is used to describe a method of extracting the road area edgesfrom the road area pixels extracted as shown in FIG. 15,

FIG. 17 shows the road image after linear approximation using the methodshown in FIG. 16,

FIG. 18 is used to describe the operation of the road region setter,

FIG. 19 shows a birds'-eye image after coordinate conversion of the roadcontour signal by the coordinate convertor in FIG. 2,

FIG. 20 is used to describe the curve patterns obtained by Houghconversion of the birds'-eye contour signal Scc in FIG. 19,

FIG. 21 is an alternative example of the curve patterns obtained byHough conversion of the birds'-eye contour signal Scc in FIG. 19,

FIG. 22 is an alternative example of the curve patterns obtained byHough conversion of the birds'-eye contour signal Scc in FIG. 19,

FIG. 23 is used to describe in-lane position detection by the in-laneposition detector,

FIG. 24 is a flow chart used to describe the overall operation of thelocal positioning apparatus shown in FIG. 1,

FIG. 25 is a block diagram of a local positioning apparatus according tothe prior art,

FIG. 26 is a block diagram of a road area extraction apparatus accordingto a second embodiment of the present invention,

FIG. 27 is an example of a brightness histogram generated from the roadi mage by the brightness histogram generator in FIG. 26,

FIG. 28 is a flow chart used to describe the overall operation of theroad area extraction apparatus shown in FIG. 26,

FIG. 29 is used to describe the method of detection lane edges from theextracted lane area by means of the road area extraction apparatus shownin FIG. 26,

FIG. 30 is a block diagram of a road area extraction apparatus accordingto a first alternative of second embodiment of the present invention,

FIG. 31 is used to describe the area occupied by the lane area in thebrightness histogram generated by the road area extraction apparatus inFIG. 30,

FIG. 32 is used to describe the brightness distribution obtained bysegmenting the road image by means of the lane image separator, and thebrightness distribution pattern obtained by the brightness distributionpattern detector in FIG. 30,

FIG. 33 is a flow chart used to describe the overall operation of theroad area extraction apparatus shown in FIG. 30,

FIG. 34 is a block diagram of a road area extraction apparatus accordingto a second alternative of the second embodiment of the presentinvention,

FIG. 35 is an example of a brightness histogram generated from the roadimage by the brightness histogram generator in FIG. 34,

FIG. 36 is a flow chart used to describe the overall operation of theroad area extraction apparatus shown in FIG. 34,

FIG. 37 is a block diagram of a road area extraction apparatus accordingto a third alternative of the second embodiment of the presentinvention,

FIG. 38 is an example of a brightness histogram generated from the roadimage by the brightness histogram generator in FIG. 37,

FIG. 39 is used to describe road image segmentation by the imageseparator 43 shown in FIG. 37,

FIG. 40 is the first part of a flow chart used to describe the overalloperation of the road area extraction apparatus shown in FIG. 37,

FIG. 41 is the second part of a flow chart used to describe the overalloperation of the road area extraction apparatus shown in FIG. 37,

FIG. 42 is a block diagram of a road area extraction apparatus accordingto a fourth alternative of the second embodiment of the presentinvention,

FIG. 43 is used to describe the relationship between extracted lane areaobtained by the road area extraction apparatus shown in FIG. 42, and atriangle formed in the image from the vanishing point and road edgepixels,

FIG. 44 is a flow chart used to describe the overall operation of theroad area extraction apparatus shown in FIG. 42, and

FIG. 45 is a block diagram of a road area extraction apparatus accordingto the prior art.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 is a block diagram of a local positioning apparatus LP accordingto a first embodiment of the present invention. This local positioningapparatus LP is used on board motor vehicles such as automobiles AM andother types of vehicles that are capable of traversing a freely definedpath of travel.

As shown in FIG. 1, the local positioning apparatus LP comprises adigital imaging apparatus 100, spatial frequency separator 200, lanearea detector 300, lane contour detector 40a, lane detector 500, andelectronic control unit (ECU) 700. Note that the ECU 700 is a devicecommonly used and known in the automobile industry, and is used todetect the vehicle condition as represented by the speed of travel andsteering condition, generate a vehicle condition signal Sc, whichincludes a velocity signal Sv and steering signal Ss, and controls thevarious electrical devices of the vehicle.

The digital imaging apparatus 100 comprises a Ph×PV pixel matrix ofimaging elements, and continuously captures images of the subject, i.e.,the area to the front in the direction of travel, where Ph is the numberof horizontal pixels and PV is the number of vertical pixels. Increasingpixel counts Ph and PV can be used to improve image resolution, and totherefore improve the positioning precision of the present invention.Increasing the pixel count, however, also increases the manufacturingcost, and the pixel count must therefore be determined by balancing therequired resolution against the manufacturing cost. While a digitalimaging apparatus 100 according to an exemplary embodiment of theinvention described below has a 428 (Ph) by 268 (PV) pixel imagingdevice, the invention shall obviously not be so limited.

The digital imaging apparatus 100 is mounted toward the front of theautomobile AM, and from this position continuously images a view Vi inthe direction of travel of the automobile AM to generate a digital imagesignal Si. This view Vi is a perspective image Vi of the area to thefront of the automobile AM as seen from the automobile AM. Furthermore,this perspective image Vi can be either a still image or a motionpicture.

The spatial frequency separator 200 is connected to the digital imagingapparatus 100, and receives therefrom the digital image signal Si. Thespatial frequency separator 200 thus extracts a low frequency componentfrom the digital image signal Si to generate a low spatial frequencysignal SL, and extracts a high frequency component to generate a highspatial frequency signal SH.

The lane area detector 300 is connected to the spatial frequencyseparator 200, and receives therefrom the low spatial frequency signalSL, based on which the lane area detector 300 detects the road surfacefrom the perspective image Vi to generate a road region signal Srr.

The lane contour detector 400 is likewise connected to the spatialfrequency separator 200, receives therefrom the high spatial frequencysignal SH, thus detects the contour of the road from the perspectiveimage Vi, and generates a road contour signal Sre. The lane contourdetector 400 also generates a region limiting signal Sr that limits thescanning area used for contour detection, and inputs this signal to thespatial frequency separator 200. More specifically, the region limitingsignal Sr is used to limit the area in the perspective image Vi that isscanned for contour detection to the area around the detected contour asa means of reducing the processing load. The spatial frequency separator200 then extracts the high spatial frequency signal SH only from thearea of the digital image signal Si specified by the region limitingsignal Sr.

The lane detector 500 is connected to the lane area detector 300 and thelane contour detector 400, and thus receives both the road region signalSrr and the road contour signal Sre. Based on the road region signalSrr, the lane detector 500 determines whether the road contour indicatedby the road contour signal Sre is correct, and generates a roadjudgement signal Sj indicating the result of this evaluation. The roadjudgement signal Sj is then fed back to the lane contour detector 400.If and only if the lane detector 500 determines that the road contourindicated by the road contour signal Sre is correct does the lanedetector 500 pass the road contour signal Sre to the local positiondetector 600.

The lane contour detector 400 updates the road contour signal Sre basedon the feedback of the road judgement signal Sj, and again outputs theroad contour signal Sre to the lane detector 500. The lane contourdetector 400 continues to update the road contour signal Sre accordingto the road judgement signal Si until the lane detector 500 determinesthat the road contour signal Sre represents a correct road contour, andthe road contour signal Sre is thus output only after it is determinedto represent a correct road contour.

The local position detector 600 thus receives the road contour signalSre from the lane detector 500, and a vehicle condition signal Sc inputfrom the ECU 700. Based on the road contour signal Sre, which thusrepresents a reliable road contour, and the vehicle condition signal Scindicative of the operating status of the motor vehicle, the localposition detector 600 detects the moving or stationary position of theautomobile AM, and thus generates a position detection signal Sp. Itshould be noted that a method and apparatus for generating an EU controlsignal for controlling vehicle operation, and navigation and controlsignals for a navigation apparatus, based on this position detectionsignal Spare disclosed in Japanese patent application H8-102892(1996-102892; filed Apr. 24, 1996) and U.S. patent application Ser. No.08/637,417 (file Apr. 25, 1996), both entitled and assigned to the sameassignee as the present application.

A simplified side view of an automobile AM in which the digital imagingapparatus 100 of the present invention is installed is shown in FIG. 4.The optical axis Ax of the digital imaging apparatus 100 is aligned withthe direction of travel Z of the automobile AM on the road, but isoriented downward from the horizontal by a known number of degrees θ.The view angle of the digital imaging apparatus lo is also wide enoughto capture the road area immediately below and in front of theautomobile AM in the perspective image Vi.

An exemplary perspective image Vi of the forward direction as seen fromthe automobile AM and represented by the digital image signal Sicaptured by the digital imaging apparatus 100 is shown in FIG. 5. It isassumed in this image that the automobile AM is travelling in the leftlane of a two lane road. The two lanes are defined by the three lanemarkers Lm1, Lm2, and Lm3. More specifically, the right lane isdelineated by lane markers Lm3 and Lm2, and the left lane by lanemarkers Lm2 and Lm1. The lane markers are commonly applied using a highvisibility paint such as white or yellow. In the image shown in FIG. 5the area at the bottom of the perspective image Vi is an image of theroad surface directly below the front of the automobile AM.

It should also be noted that the perspective image Vi captured by thedigital imaging apparatus 100 is a typical video image containing pixelsof varying luminance, density, and hue, and that line drawingrepresentations appearing as simple contour lines of objects in theviewing area are used in FIG. 5 because of simple drawing limitations.

The preferred embodiments of the present invention are described nextbelow with reference to the accompanying figures.

First Embodiment

FIG. 2 is a detailed block diagram of the spatial frequency separator200, lane area detector 300, lane contour detector 400, lane detector500, and local position detector 600 of the digital imaging apparatus100 shown in FIG. 1.

The spatial frequency separator 200 comprises a low frequency extractor202 for extracting the low frequency component of the digital imagesignal Si and generating a low spatial frequency signal SL, and a highfrequency extractor 204 for extracting the high frequency component andgenerating a high spatial frequency signal SH.

The low frequency extractor 202 uses the luminance data of the digitalimage signal Si to generate an image VL of the low spatial frequencycomponent only, and in an exemplary embodiment of the invention atwo-dimensional low pass filter.

The high frequency extractor 204 similarly use s the luminance data ofthe digital image signal Si to generate an image VH of the high spatialfrequency component only, and in an exemplary embodiment of theinvention a two-dimensional high pass filter.

More specifically, the low frequency extractor 202 and high frequencyextractor 204 separate the perspective image Vi captured by the digitalimaging apparatus 100 into a low frequency image VL and a high frequencyimage VH such that there is no overlap in the frequency components ofthe images.

The road area detector 300 comprises a lane brightness setter 302, alane image extractor 304, and a lane region setter 306. The lane areadetector 300 is upstream of the lane detector 500, which relies on aregion determined by the area detector 300 and a contour extracted bycontour detector 400. In this sense, it maybe more appropriate for thearea detector 300 to be a road surface area detector and each of theinternal components thereof to be road surface devices. It makes moresense for the lane contour detector 400 to be a "road contour detector"for contrast with the lane contour extractor 606. Hereinafter, uniformlyused are the following terms: road area detector 300, road brightnesssetter 302, road image extractor 304, road region setter 306.

The road brightness setter 302 is connected to the low frequencyextractor 202 of the spatial frequency separator 200, and to the ECU700, and receives respectively therefrom the low spatial frequencysignal SL and the vehicle condition signal Sc. Based on the velocitysignal Sv component of the vehicle condition signal Sc, the roadbrightness setter 302 sets road brightness levels Bmin and Bmax, whichdefine the luminance levels of the pixels in the area of the road in thelow spatial frequency signal SL, and outputs a road brightness signalSBr indicative of the threshold luminance together with the low spatialfrequency signal SL.

Based on the low spatial frequency signal SL and road brightness signalSBr (Bmax) supplied from the road brightness setter 302, the road imageextractor 304 extracts the road region of the low spatial frequencysignal SL, and generates a road extraction signal Srs. The road imageextractor 304 thus supplies the low spatial frequency signal SL, roadbrightness signal SBr, and road extraction signal Srs to the road regionsetter 306.

The road region setter 306 then determines what part of the low spatialfrequency signal SL represents the region in which the road is locatedbased on the road extraction signal Srs, and thereby generates the roadregion signal Srr supplied to the lane detector 500.

The lane contour detector 400 comprises a threshold setter 402, contourextractor 404, and contour extraction region limiter 406. Connected tothe high frequency extractor 204 of the spatial frequency separator 200,the threshold setter 402 sets the threshold value Eth extracted by edgedetection processing the contour points of the image representing theroad in the high spatial frequency signal SH, and outputs a contourthreshold signal Sth indicative of these threshold values Eth with thehigh spatial frequency signal SH.

Based on the contour threshold signal Sth, the contour extractor 404extracts the contour of the road image in the high spatial frequencysignal SH, and thus generates the road contour signal Sre supplied tothe lane detector 500 and to the contour extraction region limiter 406.

Based on the road contour signal Sre, the contour extraction regionlimiter 406 produces a region limiting signal Sr restricting the regionin the high spatial frequency signal SH to be used for road contourextraction, and feeds this region limiting signal Sr back to the highfrequency extractor 204 of the spatial frequency separator 200.

The high frequency extractor 204 thus extracts the high frequencycomponent and generates the high spatial frequency signal SH from anarea in the digital image signal Si restricted by the region limitingsignal Sr.

The contour extractor 404 further receives a road judgement signal Sj asfeedback from the lane detector 500. If the road contour signal Sre isdetermined by the lane detector 500 to not represent the true contoursof the lane or road, the road contour signal Sre is updated to use thecontour edge adjacent to the contour line currently extracted as thelane contour as the updated contour line, and this updated road contoursignal Sre is then output to the lane detector 500.

The lane detector 500 thus determines whether the road contour signalSre represents the correct road contour edge based on the road regionsignal Srr, and generates a road judgement signal Sj. In the exemplaryembodiment of a lane detector 500 described herein, the road judgementsignal Si is preferably high when the correct road contour edge isextracted, and is otherwise low. In other words, when the road judgementsignal Sj is low, the lane detector 500 stops outputting the roadcontour signal Sre, and causes the contour extractor 404 to update theroad contour signal Sre. As a result, the lane detector 500 outputs theroad contour signal Sre to the local position detector 600 only when theroad judgement signal Si is high.

More specifically, therefore, the contour extractor 404 detects a seriesof projected contour points representing candidates for the actual lanemarkers on the road using the high spatial frequency signal SH from thehigh frequency extractor 204. This series of projected contour points isoutput to the lane detector 500 as the road contour signal Sre. Usingthe road region data (i.e., the road region signal Srr) detected by theroad image extractor 304, and the projected contour points (i.e., theroad contour signal Sre) detected by the contour extractor 404, the lanedetector 500 then detects the contour points of the lane markers.

The local position detector 600 comprises a coordinate convertor 602,matching detector 604, lane contour extractor 606, and in-lane positiondetector 608.

The coordinate convertor 602 converts the front perspective image Vi toa birds'-eye image Vcc by applying a coordinate conversion process tothe road contour signal Sre, and thus produces a birds'-eye contoursignal Scc.

Based on the birds'-eye contour signal Scc supplied from the coordinateconvertor 602, the matching detector 604 then detects whether theroad'surface for which a contour was detected is straight or curved tooutput a matching signal Sm indicative of the matched road shape.

Based on the matching signal Sm input from the matching detector 604,the lane contour extractor 606 extracts a contour of the lane currentlyoccupied by the automobile AM to generate a lane contour signal Slc.

Next, based on the lane contour signal Slc input from the lane contourextractor 606 and the vehicle condition signal Sc supplied from the ECU700, the in-lane position detector 608 detects the position of theautomobile AM within the occupied lane to generate the positiondetection signal Sp output therefrom to the ECU 700.

Note that the local position detector 600 generates the positiondetection signal Sp using a match detection operation based, in part, onthe vehicle condition signal Sc supplied from the ECU 700. In anexemplary embodiment of the present invention as described further belowwith reference to FIGS. 20 to 22, the match detection operation of thematching detector 604 uses the steering signal SsT component of thevehicle condition signal Sc. The invention shall not be so limited,however, and other methods can be alternatively used.

The operation of the essential parts of a local positioning apparatus LPaccording to a preferred embodiment of the present invention isdescribed next below with reference to the accompanying FIGS. 5 through25. The operation of the digital imaging apparatus 100 has already beendescribed above with reference to FIG. 5. The next step is therefore todescribe the edge extraction operation of the high frequency extractor204 of the spatial frequency separator 200 with reference to FIGS. 5 and6.

As described above, the high frequency extractor 204 extracts the highfrequency component of the digital image signal Si by applying a Sobelfilter or other filtering process to the digital image signal Si. Thisfiltering operation detects the edge pixels where there is a suddenchange in pixel density. The extractor 204 then generates a high spatialfrequency signal SH representing an edge image Vh To reduce the amountof data that must be filtered for this extraction, the filteringoperation is limited to a specifically limited part of the perspectiveimage Vi. It is therefore possible to more quickly detect theperspective image edge Vh data, and generate the high spatial frequencysignal SH.

More specifically, the perspective image Vi (FIG. 5) is divided into twoparts, a top region St and a bottom-region Sb, using a single horizontalline Ls. Note that the horizontal line Ls is aligned to the A-thvertical pixel counted from the bottom of the image. The area of the topregion St is therefore Ph×(PV-A) pixels, and the area of the bottomregion Sb is Ph×A pixels. The vertical position of this A-th pixel ispreferably set to match the horizontal position of the vanishing pointof perspective image Vi when the perspective image Vi is captured withthe automobile AM on a level road. In the exemplary embodiment of theinvention, pixel A is set to the vertical position of a pixelrepresenting an image positioned approximately fifty meters away fromthe automobile AM when the perspective image Vi is captured directly bythe digital imaging apparatus 100.

Immediately after the local positioning operation of the presentinvention is started, the high frequency extractor 204 applies the abovehigh spatial frequency component extraction (filtering) operation to thebottom region Sb only to extract the edge pixels only from and generatethe high spatial frequency signal SH only for the bottom region Sb. Thisobviously means that the high spatial frequency signal SH only containsedge pixels extracted from the bottom region Sb of Ph×A pixels.

The extracted edge image Vh represented by the high spatial frequencysignal SH is shown in FIG. 6. The edge pixels of, and primarily near,the lane markers Lm1, Lm2, and Lm3 are extracted from the bottom regionSb of the perspective image Vi (FIG. 5) as the edge pixels. Though notshown in FIG. 6, it should be obvious that edge pixels from objectsunrelated to the lane markers are also extracted.

The threshold setter 402 thus receives the high spatial frequency signalSH comprising the extracted edge pixels from the low frequency extractor202, and calculates the threshold value Eth for effectively extractingthe edge pixels from the contours of the lane markers Lm in the highspatial frequency signal SH using the following equation (1).

    Eth=C*Emax+(1-C)*Emean                                     (1),

where Emax and Emean are the maximum density and the mean density valuesof the pixels on a particular horizontal line in the bottom region Sb ofthe perspective image vi; and C is a constant where 0<C<1.

The contour extraction method of the contour extractor 404 is describednext referring to FIGS. 7 and 8. An edge image Vh (SH) from which thecontour extractor 404 detects contour lines is shown in FIG. 7.

The contour extractor 404 scans the pixels in the bottom region Sb ofthe high spatial frequency signal SH using the threshold value Eth toextract the contour lines of the displayed lanes. This bottom region Sbis further divided into left and right bottom regions SbL and SbR by avertical center line Lc. Note that the vertical center line Lc risesvertically from pixel Ph/2 on the bottom line of the image in thisexemplary embodiment, but the invention shall not be so limited,

The contour extractor 404 compares the density of each pixel in thescanning area with the threshold value Eth, and extracts those pixelswith a pixel density greater than or equal to the threshold value Eth asa contour pixel. The scanning order for this operation followshorizontal line Ls and works to the left and right of the verticalcenter line La along left and right horizontal lines LhL and LhR. Afterscanning along horizontal line Ls, the contour extractor 404 drops downa particular number of pixels along the vertical center line Lc, andagain scans to right and left to obtain the contour pixels on thathorizontal line.

In exemplary practice, however, the contour extractor 404 scans first tothe left or the right of the center line in the bottom region Sb toobtain the contour pixels on that side. For example, if the left side ofthe bottom region Sb is scanned first, the contour extractor 404 scanstop to bottom starting from horizontal line Ls at pixel A, scanninglines at an interval of PLh pixels downward to the bottom of the image,and scanning every pixel right to left from the vertical center line Lcalong the left horizontal line LhL. After obtaining all contour pixelson the bottom left sector of the image in this sequence, the contourextractor 404 then returns to the top horizontal line Ls to similarlyscan the bottom right sector of the image in left to right sequence fromthe vertical center line Lc and obtain the contour pixels therein.

Skipping the horizontal scanning position at a particular interval PLhresults in extraction from an integer K number of horizontal lines Lhequivalent to the absolute value of Pv/PLh. This reduces the number ofcalculations to be performed, and thereby increases the speed of contourpixel extraction. Note that the value of PLh is a natural number set ina pixel unit increment.

When the first pixel with a particular density value is found on left orright horizontal lines LhL and LhR at a given vertical position, thatpixel Pe is defined as a contour pixel, and contour pixel scanning forthat horizontal line is skipped. Scanning then advances directly to thenext horizontal line PLh pixels down.

In other words, contour pixels are detected at the inside edges of thelane markers Lm1 and Lm2 on the left and right sides of the shown laneat the first horizontal line Lh counted down from the horizontal lineLs, as shown in FIG. 7. At the third and fourth lines, however, there isa gap between successive lane markers Lm2 dividing the right and leftlanes, and contour pixel detection to the right side of the verticalcenter line Lc at these lines, or more specifically, at any verticalposition between these lane markers Lm2, will fail to detect a contourpixel from any lane marker Lm2 and may detect a contour pixel from thelane marker Lm3 at the edge of the adjacent lane. The edges of markingsother than the lane markers but causing a similar sudden change in pixeldensity, e.g., soil or even imaging flaws, will also be detected ascontour pixels. Such contour pixels are, however, irrelevant todetecting the vehicle lanes, and are therefore simply a noise componentinsofar as the position detection operation of the present invention isconcerned. As a result, these noise components are removed by the lanedetector 500 as described in further detail below.

An extracted contour image Vre represented by the road contour signalSre obtained as described above from the image in FIG. 7 is shown inFIG. 8. The detected contour pixels of the lane markers Lm1, Lm2, andLm3 are shown as contour signals ScL, ScR, and ScR' where ScR' is noisedata. Contour pixel signal ScL' represents noise data extracted due tosome other flaw, shadow, or marking apparent in the image.

The method of limiting the detection area by means of the contourextraction region limiter 406 is described next below with reference toFIGS. 9 and 10. As shown in FIG. 9, the contour extraction regionlimiter 406 defines a limited region RsL and RsR of a known width Wr tothe right and left sides of the extracted contour signals ScL and ScRbased on the road contour signal Sre. The region limiting signal Sridentifies these limited regions RsL and RsR, and is output to the highfrequency extractor 204 of the spatial frequency separator 200. However,when the contour extractor 404 is unable to extract the lane contour andan error signal see (not shown in the figures) in place of the roadcontour signal Sre, the extraction region is not limited.

The width Wr of the limited region is determined with respect to theamount of movement in the x-axis (horizontal) direction of the lanemarkers in the image as a result of lateral movement of the automobileAM. More specifically, Wr is the distance that lane marker Lm1 can movein the perspective image Vi in one system cycle, i.e., 33 mS in anexemplary embodiment of the invention. It therefore follows that thelimit width wr increases towards the bottom of the image.

As shown in FIG. 10, the high frequency extractor 204 limits the areafor edge extraction in the bottom region Sb of the perspective image Vito the limited regions RsR and RsL based on the region limiting signalSr. In addition to reducing the amount of data to be processed andtherefore increasing the processing speed, this process also suppressesnoise components from outside the lane contours, and therefore improvesthe lane tracking ability.

When a noise component is sufficiently removed from the limited regionsRsR and RsL, as is the contour line Scr' of the opposing lane marker Lm3described above, the noise component is removed by the contourextraction region limiter 406. However, when a contour noise componentis near a lane contour, as is the contour line ScL' in FIG. 8, the noisecomponent will be within the limited region Rsr. In this case the noisecomponent cannot be removed by the contour extraction region limiter406. As a result, these noise components must be removed by the lanedetector 500, as will be described later below.

The method of generating a low spatial frequency image by means of thelow frequency extractor 202 of the spatial frequency separator 200 isdescribed next below with reference to FIG. 11.

The low frequency extractor 202 generates the low spatial frequencysignal SL by low-pass filtering the digital image signal Si. The subjectof the perspective image Vi appears as a fuzzy grouping of pixels ofvarious luminance levels in the low frequency image VL represented bythe low spatial frequency signal SL. The low frequency image VLrepresented by the low spatial frequency signal SL is thus a luminanceimage. To reduce the processing load of the low spatial frequency signalSL generating operation, the luminance pixels are therefore extractedfrom a limited part of the perspective image Vi, specifically, the Ph×Apixel bottom region Sb. This is similar to the method of generating thehigh spatial frequency signal SH described above.

The luminance of the pixels in the road region of the low frequencyimage VL tends to increase with the distance from the automobile AM,i.e., the distance from the digital imaging apparatus 100. The same istrue of the luminance level of pixels in image areas corresponding tothe road shoulder, drainage channels, and guard rails. In the image inFIG. 11, the pixels at vertical pixel level A, i.e., along horizontalline Ls, have the highest luminance level of any pixels associated withthe same object, and the pixels toward the bottom of the image have thelowest luminance level.

The luminance level of pixels at the same vertical height but associatedwith different objects will, however, vary from object to object. Morespecifically, the luminance will vary according to the reflectance ofthe subject material, and the angle of incidence of light from the lightsource. The above described relationship between luminance and distance,i.e., that luminance increases with distance, is, however, retained.

The low frequency image VL is thus produced only from the bottom regionSb of the perspective image Vi obtained by the digital imaging apparatus100 imaging primarily the area directly in front of the vehicle.Furthermore, a majority of the pixels in the low frequency image VLrepresent the road surface, and this tendency is therefore substantiallyapplicable to all pixels in the low frequency image VL. The pixels forthe road surface detected along the horizontal line Ls at the farthestpoint from the digital imaging apparatus 100 in the low frequency imageVL are therefore defined as the farthest road pixels Pf, and theluminance level of those pixels is defined as maximum luminance Bmax.The pixels for the road surface detected at the bottom of the imagenearest the digital imaging apparatus 100 in the low frequency image VLare therefore defined as the nearest road pixels Pn, and the luminancelevel of those pixels as minimum luminance Bmin.

Note that the location of the farthest road pixels Pf and nearest roadpixels Pn can be freely set as required.

A histogram of all pixels in the low frequency image VL is shown in FIG.12 with luminance levels plotted on the horizontal axis and the numberof pixels with a given luminance level plotted on the vertical axis. Aswill be known from FIG. 12, a majority of the pixels in the lowfrequency image VL are road surface pixels with luminance betweenminimum luminance Bmin and maximum luminance Bmax. The luminance rangebetween minimum luminance B-min and maximum luminance Bmax is thereforecalled the road luminance range Br, and the pixels in this roadluminance range Br are called road pixels Pr.

When the digital imaging apparatus 100 is installed S at substantiallythe front center of the automobile AM as described with reference toFIG. 4, the positions of the farthest road pixels Pf and nearest roadpixels Pn are set at appropriate vertical positions Vpf and Vpn in theimage on the vertical center line Lc. The road surface occupying thearea between Vpf and Vpn is then indicated by the road pixels Pr. Theposition of Vpf can therefore be thought of as the farthest distance atwhich the road surface is detected, and the position of Vpn as thenearest distance at which the road surface is detected.

The position of the farthest road pixels Pf is set appropriately on thefarthest road detection distance Vpf according to the position at whichthe digital imaging apparatus 100 is mounted on the automobile M4, andthe position of the nearest road pixels Pn is likewise set appropriatelyon the nearest road detection distance Vpn.

The farthest road detection distance Vpf and nearest road detectiondistance Vpn can be set according to the vehicle speed and theprocessing capacity (speed) of the local positioning apparatus LP asdescribed further below.

The relationship between the luminance of only road pixels in the lowfrequency image VL and the distance of the road surface from which thepixels are captured is shown in FIG. 13. The values shown in FIG. 13 arefor pixels in an image experimentally obtained while driving on astraight section of highway on a sunny day with the digital imagingapparatus 100 mounted on the front of the automobile AM. The horizontalaxis indicates the horizontal scanning position (Pv) in the lowfrequency image VL, i.e., the relative distance D of each pixel from thedigital imaging apparatus 100, and the vertical axis indicates theluminance B at that distance. That the distance-luminance relationshipis not a flat line but has a right ascending slope is a result of lanemarkings, soiling, and color and luminance spots caused by reflectancefrom obstructions on the road. As already described above, the arearepresented by a single pixel increases as the horizontal scanningposition is elevated, meaning that the scanning position becomes moredistant from the image pickup 100. However, the relationship betweenimaging distance and pixel luminance can be expressed by equation (2)below.

    B=A*D+C                                                    (2)

Note that values of A=0.4128 and D=108.56 have been experimentallyobtained on straight lanes of an expressway.

By correctly setting C' and C" so that C'>C>C", and obtaining parallellines of:

    B=A*D+C'                                                   (3),

    and

    B=A*D+C"                                                   (4),

above and below the line represented by equation (2), the range ofluminance values of pixels at a particular scanning position can beobtained.

The method whereby the road brightness setter 302 determines theluminance level of the road surface is described next below withreference to FIG. 14. Based on the relationship between the imagingdistance and pixel luminance described with reference to FIGS. 11 to 13,the farthest road pixels Pf and nearest road pixels Pn in the lowspatial frequency signal SL are set to the pixel positions correspondingto the distance defining the road area to be used for positioningdetection. The pixel luminance at the farthest road pixels Pf is thendefined as maximum luminance Bmax, and the pixel luminance at thenearest road pixels Pn is defined as the minimum luminance Bmin togenerate the road brightness signal SBr. Note that the road brightnesssignal SBr may be set to have luminance levels in the range defined byequations (3) and (4) above. The locations of the farthest road pixelsPf and the nearest road pixels Pn in the low frequency image VL areexpressed as Pf(xPf,yPf) and Pn(xPn,yPn) using the coordinate systemshown in FIG. 3.

By setting the position of the farthest road pixels Pf from which themaximum luminance Bmax is detected to an appropriate position accordingto the processing speed Tp of the local positioning apparatus LP and thespeed of the automobile AM, high precision lane detection is possiblebased on the operating conditions of the automobile AM on which thelocal positioning apparatus LP is installed.

For example, by moving the farthest road pixels Pf vertically upward inthe low frequency image VL according to the velocity signal Sv of theautomobile AM, the detection precision at points far from the automobileAM can be maintained even when the automobile AM is travelling at a highrate of speed. When the automobile AM is travelling slowly, lanedetection precision can be improved by moving the position of thefarthest road pixels Pf closer. Because the vehicle will pass thedetection point while the detection process is in progress when thevehicle is travelling fast and the farthest road pixels Pf is close, itis preferable to detect the maximum luminance Bmax at a suitabledistance from the vehicle based on the velocity signal Sv and theprocessing capacity of the local positioning apparatus LP.

When the automobile AM is travelling on a straight road, the farthestroad pixels Pf can be set on the vertical center line Lc of the lowfrequency image VL. However, when the automobile AM is on a curvingroad, setting the farthest road pixels Pf on the vertical center line Lcmay result in the maximum luminance Bmax being extracted from a guardrail or other non-road subject within the low frequency image VL.

In such cases, it is necessary to estimate the curvature of the roadfrom the steering angle of the automobile AM so that only pixels withinthe image area of the road surface in the low frequency image VL areused for extracting the maximum luminance Bmax. This can be accomplishedby moving the farthest road pixels Pf horizontally within the lowfrequency image VL according to the steering signal Ss so that thefarthest road pixels Pf are within the road surface area.

By moving the position of the farthest road pixels Pf freelyhorizontally and vertically within the low frequency image VL based onthe velocity signal Sv and steering signal Ss, the farthest road pixelsPf can be adjusted to the road image area. Note that a value other thanthe velocity signal Sv can be alternatively used, including a distancevalue (vertical image position value) entered by the user. Data on thedirection of travel (straight or curved) supplied by a navigation systembased, for example, on map data, can also be used in place of thesteering signal Ss. More specifically, the pixels of the subject areacan be reliably captured by moving or fixing the positions of thefarthest road pixels Pf and nearest road pixels Pn in the low frequencyimage VL as necessary.

The method whereby the road image extractor 304 extracts an image of theroad surface is described next below with reference to FIGS. 15 to 17.The road image extractor 304 scans the pixels in the bottom region Sb ofthe low spatial frequency signal SL based on the pixel luminancerelationship to the scanning position shown in FIG. 13 and defined byequations 2, 3, and 4 to extract the pixels in the road image area. Theroad image contours extracted by the road image extractor 304 aresimilar to the contour lines of the lanes obtained by the contourextractor 404 and described with reference to FIG. 12. The road imageextractor 304, however, extracts the pixels with a luminance levelbetween maximum luminance Bmax and minimum luminance Bmin, i.e., pixelssatisfying equations 2, 3, and 4 at each horizontal scanning positionPv, rather than using an edge extraction method as does the contourextractor 404.

Region pixel sets Elm, Erm, and Erm' corresponding to the contour pixelsScL, ScR, and ScR' in FIG. 8 are obtained at the inside circumferencepart of lane markers Lm1, Lm2, and Lm3. It should be noted that linesrepresenting the inside edges of the lane markers Lm1₇ Lm2, and Lm3 canbe obtained as necessary based on the region pixel sets Elm, Erm, andErm' using a least squares method or other suitable method such as theHough conversion described further below with reference to FIGS. 20 and22. An example of Hough conversion of pixel set Elm is shown in FIG. 16,and an example of a linear approximation of the road surface is shown inFIG. 17. Note that linear approximation effectively removes the insideedge part set Erm' of the lane marker Lm3 of the adjacent lane.

The method whereby the road region setter 306 defines the road surfacearea is described next below with reference to FIG. 18.

The road region setter 306 sets road edge areas AeL and AeR based on theroad extraction signal Srs by adding a known horizontal width Wa to bothright and left sides of the extracted inside edge areas Elm and Erm. Theroad region signal Srr output to the lane detector 500 is indicative ofthese road edge areas AeL and AeR. When the road region setter 306cannot extract the road area and the error signal See is output in placeof the road extraction signal srs, the road edge areas AeL and AeR arenot set. Note that while the operation of the road image extractor 304is similar to that of the contour extraction region limiter 406, thelimit width Wa can be an appropriately defined value based on the valuesC, C', and C" shown in equations 2 to 4 above.

Using the low spatial frequency signal SL from the low frequencyextractor 202, the road image extractor 304 divides the low frequencyimage, or luminance image, VL into areas of pixels having a particularrelationship based on the luminance distribution. Certain road knowledgeis then referenced and applied to these divided regions to detect theroad area.

The "road knowledge" used herein includes knowing that the areacontaining the farthest road pixels Pf and nearest road pixels Pn in theperspective image Vi captured by the digital imaging apparatus 100 iswithin the road area. Other knowledge includes knowing that thevanishing point at which the right and left lane markers intersect doesnot change suddenly, and that the pixel area of the road surface issubstantially constant. The area containing the nearest road pixels Pnand the adjacent area can therefore be defined as the road area.

The operation of the lane detector 500 is described next.

The lane detector 500 determines whether the contour lines ScR and ScLdescribed by the road contour signal Sre from the lane contour detector400 (FIG. 8) are within the road edge areas AeR and AeL described by theroad region signal Srr. To accomplish this the lane detector 500determines whether the ScR value is within AeR±Wa at the same horizontalscanning line position (distance). If ScR is within this range, the roadcontour signal Sre is determined to represent a correct road contour.The road judgement signal Sj is therefore output low, and the roadcontour signal Sre is passed through to the local position detector 600.

However, if ScR is outside the range AeR±Wa, the road contour signal Sreis determined not to represent a correct road contour, but to be withina noise contour component. The road judgement signal Sj is thereforeoutput high, and the road contour signal Sre is not output.

It is therefore possible as described above to produce a low frequencyimage and a high frequency image with no spatial frequency overlapbetween the images from the same luminance image data, and separatelydetect from these images the road area and lane marker contour points.As a result, lane markers can be detected with high precision.

The coordinate conversion principle used by the coordinate convertor 602to convert the extracted contour image Vre is described briefly belowwith reference to FIGS. 3 and 4.

As described above, the coordinate convertor 602 converts the coordinatesystem of the road contour signal Sre to convert the extracted contourimage Vre represented by the road contour signal Sre to a birds'-eyeimage Vcc. However, the perspective representation of objects in theextracted contour image Vre results in shape distortion that increaseswith the distance from the digital imaging apparatus 100. This meansthat, for example, when there are two identical objects with one placedat a greater distance from the digital imaging apparatus 100, thefarther object will appear smaller than the nearer object. This shapedistortion increases with the distance from the digital imagingapparatus 100 or automobile AM.

The coordinate system of perspective images Vi, Vh, and Vre is shown inFIG. 3. The birds'-eye view coordinate system referenced to theautomobile AM is shown in FIG. 4. The x-axis of the image coordinatesystem (X,y) is parallel to and oriented in the same direction as theX-axis in the coordinate system of the birds'-eye image Vcc, and they-axis is inclined to the Y-axis θ degrees because the optical axis Axof the digital imaging apparatus 100 is inclined θ degrees from level.The X-axis and Z-axis therefore define a level plane Pd that is parallelto the travelling surface, i.e., road surface, of the automobile AM. TheY-axis and y-axis are aligned, and the X-axis is perpendicular to thesurface plane of FIG. 4. The origin O matches the origin of thecoordinate system of the extracted contour image Vre. The optical axisAx of the digital imaging apparatus 100 passes through the origin O.

The coordinates of the extracted contour image Vre can be converted tothe coordinates of the birds'-eye image Vcc using the followingequations.

    X=(x/F)(Z cos θ-Y sin θ)                       (5),

    Z=Y(F cos θ+Y sin θ)                           (6),

    and

    Y=-H                                                       (7),

where F is the focal distance of the optical lens of the digital imagingapparatus 100; θ is the angle between the Z-axis (horizontal axis) andthe optical axis Ax of the lens; and H is the distance from the roadsurface to the origin of the optical axis of optical lens. The values ofθ and H are preferably set to obtain a perspective image Vi of the areato the front of the automobile AM as described with reference to FIG. 5.In an exemplary embodiment of the present invention, θ is 6 degrees, andH is 1.2 meters.

The birds'-eye contour signal Scc is then obtained by converting thecoordinates of the road contour signal Sre by converting the distancesin the extracted contour image Vre using the above equations 5 to 7.Horizontal distances in this birds'-eye contour signal Scc areaccurately represented irrespective of the Z-axis direction from thedigital imaging apparatus 100. More precisely, the coordinate-convertedimage Vcc is not a birds'-eye image, but is a plan view of the roadsurface captured from a plane parallel to the road surface. Furthermore,image Vcc always represents the road surface as a flat surface evenwhen, for example, there are low rises or obstructions in the road. Thiserror/difference (low rises or obstructions), however, does not degradethe detection precision of the local positioning apparatus according tothe present invention. This is because the positioning detectionoperation of the present invention uses a relatively near distance ofonly approximately fifty meters forward as the farthest forward pointused for detection and control, and bumps and obstructions on the roadsurface can therefore be ignored.

If a full birds'-eye image of the road surface, i.e., a plan view imageof the road surface captured perpendicularly to a horizontal planeperpendicular to the vertical line with respect to the gravitationalaxis is required, it can be obtained by various methods. One such methodis to not fix the optical axis Ax of the digital imaging apparatus 100to the automobile AM, and use a gyroscope or other automatic attitudecontrol device to always obtain the image Vi from a constant attitude tothe horizontal.

For reference, reverse coordinate conversion from a birds'-eye viewcoordinate system to a perspective view coordinate system can beachieved using equations 8 and 9 below.

    x=FX/(Z cos θ+H sin θ)                         (8)

    z=F(H cos θ+Z sin θ)/(Z cos θ-H sin θ)(9)

The contour image Vcc represented by the coordinate-converted birds'-eyecontour signal Scc is shown in FIG. 19. The lane markers Lm1, Lm2, andLm3 are represented by corresponding edge contours ScL, ScR, and ScR'.As will be known by comparison with the extracted contour image Vre inFIG. 8, the edge contours ScL, ScR, and ScR' are parallel throughoutFIG. 19., and represent the actual lanes on the road. In other words,the coordinate-converted birds'-eye contour signal Scc contains correctdimension information for the subject.

Based on the coordinate-converted birds'-eye contour signal Scc, thematching detector 604 connected to the coordinate convertor 602 obtainsline segments or arcs matching the edge contours ScL, ScR, and ScR' byapplying the following equations.

The first step is a Hough conversion of the pixel data for the edgecontours ScL, ScR, and ScR' of the lane markers in the birds'-eyecontour signal Scc. This is accomplished by applying equation (10) belowseparately to the corresponding contour lines.

    ρ=X cos φ+Z sin φ                              (10)

where ρ is the distance between the origin O and the pixel in the Z-Xcoordinate system, and φ is the angle between the X-axis and a linejoining the origin O and a particular pixel. The following equation (11)is then obtained after calculating equation (10).

    X=(ρ-Z sin φ)/cos φ                            (11)

A group of curves is obtained for each contour line by scanning thebirds'-eye contour signal Scc and converting the contour line data toparametric space based on the above Hough conversion. The method ofdetermining whether a contour line is a straight line segment or arcbased on this group of curves is described below with reference to FIGS.10 to 22.

Typical patterns obtained when the Hough-converted line data is astraight line is shown in FIG. 20. Note that ideally each of the curvesintersect at a single point Cp. As a practical matter, however, the lanemarkers Lm and contour lines are not usually completely straight lines,and convergence at a single point as shown in FIG. 20 thus rarelyoccurs. This problem can be resolved by noting that the curves tend tointersect at the same point, and investigating the frequency Fc thateach curve intersects at each point (pixel) in the parametric space. Ifthis frequency Fc is greater than a particular threshold value Eth, thatpoint is defined as the unique point of intersection cp. Thecorresponding contour line is then defined as a straight line, and theappropriate equation for a line is obtained.

Typical patterns obtained when the Hough-converted line data is a curveis shown in FIGS. 21 and 22. As shown in these figures, there is neitherno point of intersection (FIG. 21), or a group of curves intersect at aplurality of points Cpn (Cp1 to Cp6) as shown in FIG. 22. Note that n inthis case is an integer value. The frequency of intersection Fc is againobtained. If there is no point where a new frequency threshold Fth' isexceeded, the group of curves is determined to not intersect, and thecorresponding contour line is determined to be a curve (arc). Theappropriate equation for a curve is then obtained.

It should be noted that these equations make it possible to determineparticular dimensional features of the road, i.e., the lane that theautomobile AM is either following or stopped in.

As shown in FIG. 2, the lane contour extractor 606 is connected to thematching detector 604, and receives therefrom the matching signal Sm.Note that the matching signal Sm also contains contour line data ScR',which is obtained from noise components as described above.

The lane contour extractor 606 compares the dimensional features datacontained in the matching signal Sm with particular predetermined datasuch as the vehicle width, lane width, and the pattern of the centerline lane marker Lm2 to remove contour data ScR' that is not appropriateto the target lane. The remaining contour line data ScR and ScLcorresponding to the current lane is output as the matching signal Sm.If there is no data corresponding to the lane in the matching signal Sm,an error signal See (not shown in the figures) is output.

The filtering operation of the lane contour extractor 606 is describednext with reference to FIG. 19.

This birds'-eye image Vcc contains three contour lines ScL, ScR, andScR'. Counting down from the top of the image, the left two contourlines ScL and ScR are a contour line pair describing a single lane Lmdown to the second pixels. The center line lane marker Lm2 in thisimage, however, is a divided line, and the contour line ScR' at the farside of the adjacent lane is therefore extracted from the birds'-eyecontour signal Scc at pixels 3 to 5. The distance between ScL and ScR'in this case is obviously greater than the width of a single lane, butit is still not known at this point whether contour line ScL or ScR'represents noise.

From pixel 6 to 11, however, contour lines ScL and ScR are obviouslypaired. The left contour line ScL is also determined by the matchingdetector 604 to match a single arc ScLm, and pixels 3 to 5 furthermorecorresponding to lane marker Lm1. The three pixels from pixel 3 to 5 incontour line ScR' are therefore ignored as noise. The two contour linesScL and ScR are therefore extracted as a correct contour line pair. Thematching contour lines ScLm and ScRm are thus selected, and output withthe information of selected contour lines as the lane contour signalSlc.

Combined with the road area evaluation result output as road contoursignal Sre by the lane detector 500, the lane contour extractor 606 thuseffectively eliminates noise components, such as other markings andsoiling on the road, that are unrelated to lane definition from the lanecontour signal Slc.

The extracted lane contour image Vle represented by the road contoursignal Sle after noise removal by the lane contour extractor 606 isshown in FIG. 23.

The operation of the in-lane position detector 608 is described nextwith reference to FIG. 23.

Using the lane contour signal Slc, the in-lane position detector 608obtains the relationship between a circle or line tangential to orintersecting the right and left contour lines ScLm and ScRm to obtainthe points of intersection HL and HR at Z=0. The coordinates for themidpoint of this line segment, i.e., the coordinates (HC,0) of the linesegment HL-HR, are then obtained. Note that the camera is located at(0,0). Therefore, if the digital imaging apparatus 100 is mounted at thewidthwise center of the automobile AM, the position of the automobile AMrelative to the lane can be expressed as the lateral displacement -HCfrom the lane center.

The overall operation of a local positioning apparatus LP according toan exemplary embodiment of the present invention is described next belowwith reference to the flow chart in FIG. 24.

Operation starts after a digital image signal Si is generated by meansof the digital imaging apparatus 100 capturing a picture of the view tothe front of the automobile AM (block #1).

The spatial frequency separator 200 then extracts the low spatialfrequency signal SL and high spatial frequency signal SH from thedigital image signal Si (block #3).

The threshold setter 402 then sets a threshold value Eth based on theedge pixel density in the high spatial frequency signal SH, andgenerates the contour threshold signal Sth (block #5).

Based on the contour threshold signal 5th, the contour extractor 404extracts the lane marker contours using the threshold value Eth derivedfrom the high spatial frequency signal SH, and outputs the road contoursignal Sre (block #7).

Whether an error signal See is output in place of the road contoursignal Sre is then determined (block #9). If the error signal See isoutput, contour lines have not been extracted. A YES is thereforereturned and the procedure loops back to block #1. If NO is returned,the procedure advances.

Based on the road contour signal Sre, the contour extraction regionlimiter 406 then defines the area in the high spatial frequency signalSH to be used for contour extraction, and outputs a region limitingsignal Sr to the high frequency extractor 204 (block #11). This feedbackloop (see FIG. 2) causes the high frequency extractor 204 to extract thehigh frequency component in the area defined by the region limitingsignal Sr within the digital image signal Si at block #3 above, andgenerate the high spatial frequency signal SH from this limited region.

The road brightness setter 302 then sets a road surface luminance area(Bmax, Bmin) in the low spatial frequency signal SL resulting from block#3, and outputs a road brightness signal SBr (block #13).

Based on the road brightness signal SBr, the road image extractor 304then extracts the road surface image from the low spatial frequencysignal SL to generate the road extraction signal Srs (block #15).

Next, based on the road brightness signal SBr and road extraction signalSrs, the road region setter 306 generates a road region signal Srrdescribing the road edge areas AeR and AeL (block #17).

Whether the error signal See is output in place of the road regionsignal Srr is then detected (block #19). If the road region has not beendetected, YES is returned and the procedure loops back to block #1. IfNO is returned, the procedure advances.

The lane detector 500 then determines whether the extracted contourlines described by the road contour signal Sre are valid based on theroad region signal Srr (block #21). If the result is NO, the contourextractor 404 is driven again by means of the road judgement signal Sjto update the road contour signal Sre. The sequence from block #7 toblock #21 is thus repeated until valid contour lines are extracted. Theroad contour signal Sre is passed to the local position detector 600only once the contour lines are determined to be valid (correct).

The coordinate convertor 602 then coordinate converts the extractedcontour image Vre of the road contour signal Sre to a birds'-eye imageVcc to generate a birds'-eye contour signal Scc (block #23).

The matching detector 604 then uses a Hough conversion or otherappropriate conversion method to match an equation describing theinternal contour lines of the birds'-eye contour signal Scc to straightline segments or arcs (block #25). The matching detector 604 alsooutputs a matching signal Sm representing the matching line segments orarcs.

Whether an error signal See is output in place of the matching signal Smis then determined (block #27). If there are no matching line or arcsegments, YES is returned and the procedure loops back to block #1. IfNO is returned, the procedure advances.

Dimensional features of the matching contour lines indicated by thematching signal Sm are then compared with the dimensional features ofthe automobile AM by the lane contour extractor 606 (block #29). Thelane contour extractor 606 thus extracts a pair of lane contour linesdescribing the lane currently occupied by the vehicle, and generates alane contour signal Slc.

The current position of the automobile AM is then detected by thein-lane position detector 608 based on the lane contour signal Slc, anda position detection signal Sp is thus output (block #31).

Note that, if the error signal See is detected in blocks #9, #19, or #27above, the procedure is returned to the beginning (block #1) to achievemore precise lane detection and position detection by executing theprocedure for each intervening block based on the high spatial frequencysignal SH and low spatial frequency signal SL extracted at the sametime. Depending on the desired precision, however, blocks #9, #19, or#27 can be omitted without substantially affecting the results achievedby the present invention.

Second embodiment

A local positioning apparatus according to a second exemplary embodimentof the present invention is described next below with reference to FIGS.26 to 45, FIG. 26 is a block diagram of a road area extraction apparatusaccording to the present embodiment of the invention, which comprises adigital image pickup 11, brightness histogram generator 12, lanebrightness separator 13, and lane polygon detector 14.

The digital image pickup 11 is typically a video camera used forcapturing an image of the road in front of the vehicle. If the digitalimage pickup 11 is mounted at approximately the center front of thevehicle, the bottom center portion of the captured image willnecessarily contain the road surface, as is described further below.

The brightness histogram generator 12 produces a brightness histogram,as shown in FIG. 27, from the input road image data. Using thisbrightness histogram, the brightness histogram generator 12 also detectsthe vehicle position in the image, i.e., detects the luminance at thebottom middle part of the image.

Using the histogram, the lane brightness separator 13 detects the changein brightness to the right and left sides of the luminance at thevehicle position in the image. By detecting valleys in the brightnesslevel, the lane brightness separator 13 finds the luminance rangecorresponding to the lane area, and extracts the pixels in that range aslane area candidate pixels.

Referencing the extracted lane area candidate pixels, the lane polygondetector 14 finds the edges of the corresponding pixel area to the rightand left sides of the bottom scanning line of the image. It then detectsthe midpoint between these side edges, and then repeats the edge pixeldetection process at another scanning line higher in the image. Theresult of this process is a polygon that is detected as the road area.

This process is described more specifically below with reference to theflow chart in FIG. 28.

The first step is to generate a brightness histogram from the luminancedata captured by the digital imaging pickup 11 (step S1).

Using the brightness b0 at the bottom center of the image, the positionin the histogram corresponding to the road surface directly below thevehicle is obtained (step S2). A valley on the left side is detectedfirst.

Mean brightness values b_(mean) 1 and b_(mean) 2 are then obtained forthe overlapping brightness regions b₀ -b₁ ≦b<b₀ and b₀ -3b₁ /2≦b<b₀ -b₁/2 where b is a brightness parameter and b₁ is a constant brightnesslevel (step S3).

The equation b_(mean) a=b_(mean) 2-b_(mean) 1 is then calculated (stepS4), and the sign of b_(mean) a is then determined (step S5). A valleyis detected by determining when the sign changes from negative topositive.

If the sign has not changed from negative to positive, b₀ =b_(o) -b₁ /2(step S6), and the procedure loops back to step S3 (step S6). Thisprocess is repeated until a valley is detected to the left side.

A similar process is then executed to detect a valley on the right. Thefirst step (step S7) is to reinitialize b₀ to the value detected in S2,and then calculate b_(mean) 3 and b_(mean) 4 for the regions b₀ <b≦b₀+b₁ and b₀ +b₁ /2<b≦b₀ +3b₁ /2. The equation b_(mean) b=b_(mean) 4b_(mean) 3 is then obtained (step S8).

A valley is detected by determining when the sign changes from negativeto positive. If the sign has not changed from negative to positive, b₀=b₀ +b₁ /2 (step S10) and the procedure loops back to step S7 (step S9)This process is repeated until a valley is detected to the right side.

The pixels in the brightness region between the detected valleys arethen detected as the lane area candidate region of the image (step S11).

Referencing the extracted lane area candidate pixels, the edges-of thepixel areas to the right and left sides of the bottom scanning line ofthe image are detected as shown in FIG. 29. It then detects the midpointbetween these s ide edges, and then repeats the edge pixel detectionprocess at another scanning line higher in the image. The result of thisprocess is a polygon as shown in FIG. 29 detected as the road area.

As described above, the road area extraction apparatus of the presentembodiment detects an area containing the luminance value directly belowthe vehicle in the brightness histogram. Using the image areacorresponding to this brightness value, the edges of the road area arethen detected using a center-outward pixel comparison method. The roadedges thus extracted are an accurate representation of the road area inthat image.

A first alternative of second embodiment of a road area extractionapparatus according to the present invention is described next belowwith reference to FIGS. 30 and 31.

This road area extraction apparatus comprises a digital image pickup 21,brightness distribution pattern detector 22, and lane image separator23.

The digital image pickup 21 is typically a video camera used forcapturing an image of the road in front of the vehicle. If the digitalimage pickup 21 is mounted at approximately the center front of thevehicle, the bottom center R portion of the captured image willnecessarily contain the road surface, as is described further below.

The brightness distribution pattern detector 22 samples a defined regionS₀ from the bottom middle of the road image data as shown in FIG. 31,and determines the brightness distribution a in this area.

The lane image separator 23 then divides the lane image and compares thebrightness distribution b of each image segment with the brightnessdistribution pattern a obtained by the brightness distribution patterndetector 22 as shown in FIG. 32. If the patterns match, the image areais determined to be a road area.

The operation of this road area extraction apparatus is described belowwith reference to the flow chart in FIG. 33.

The brightness distribution pattern of the defined region S₀ at thebottom middle of the road image data is obtained by calculatingbrightness mean m and distribution s in region S₀. The maximumlogarithmic probability l is then calculated as follows where n is thenumber of data points in the region S₀.

    1(μ, φ2)=-(n/2)log 2πφ2-n/2                  (12)

The road image is then separated into N×N (the number of data points inthis region), the brightness distribution pattern of an area Sidifferent from region So is obtained by calculating brightness mean μiand distribution φi in region Si. The maximum logarithmic probability liis then calculated as follows where n is the number of data points inthe region Si.

If the maximum logarithmic probability li is 1-α≦li≦1+α (where α is apositive integer) in step S3, the area is determined to be a lane instep S4. For example, region S1 is determined to be a lane because thedistribution patterns are similar although the brightness levels ofpatterns a and b in FIG. 32 differ.

If the above equation is not true, the area is determined not to be alane at step S5.

If i≧N×N at step S6, the process ends. If not, i is incremented to i+1at step S7, and the procedure loops back to step S2.

As described above, the road area extraction apparatus of the presentembodiment detects an area with the same brightness pattern as the areadirectly below the vehicle in the brightness histogram to accuratelyextract the lane area. The overall image can also be segmented based ona probability comparison with the S0 of the brightness pattern.

A second alternative of a road area extraction apparatus according tothe second embodiment of the present invention is described next belowwith reference to FIGS. 34 and 35. This road area extraction apparatuscomprises a digital image pickup 31, brightness histogram generator 32,and lane image extractor 33.

The digital image pickup 31 is typically a video camera used forcapturing an image of the road in front of the vehicle as described inthe preceding aternatives.

The brightness histogram generator 32 produces a brightness histogramfrom the input road image data. Using this brightness histogram, thebrightness histogram generator 32 also detects the vehicle position inthe image, i.e., detects the luminance at the bottom Piddle part of theimage. In this histogram, the number of pixels having a brightness levelgreater than the brightness at the vehicle position and corresponding tothe road area is substantially constant. This pixel count is defined aspixel count g below. Note that this pixel count g also varies accordingto such factors as the camera height and view angle.

The lane image extractor 33 determines the brightness range containingpixels corresponding to the road. The pixels of the road image in thebrightness range determined to be the lane area of the brightnesshistogram are extracted to obtain the lane area.

The operation of the above road area extraction apparatus is describedbelow with reference to the flow chart in FIG. 36.

The first step is to generate a brightness histogram from the luminancedata captured by the digital imaging pickup (step S1). The brightnesshistogram shown in FIG. 35 is exemplary of this histogram.

Using the brightness b0 at the bottom center of the image, the sum s ofthe frequency, or number of pixels observed is obtained (step S2).

Using the frequency a of pixels with a higher brightness level, the sums of the frequency is obtained (step S3).

The lane area is determined to have been extracted if s is greater thanthe previously obtained pixel count g corresponding to the lane area. Ifs is less than g, the procedure loops back to step S5 and S3, and thefrequency ai of brightness b0+Db is obtained. If s is greater than g,the procedure advances to step S6 and S7. The brightness b1 and originalbrightness b0 at the vehicle position are then used to extract thepixels of brightness br where b0≦br≦b1.

These pixels are then located on the road image to obtain the lane area(step S7).

The brightness values of the road use the common tendency of a road tohave a continuous brightness level that is lower than the brightnessimmediately before the vehicle. The brightness immediately below thevehicle is therefore obtained from a brightness histogram, thebrightness values containing the frequency of the number of pixels inthe lane area of the image, which is previously determined, is obtainedfrom this brightness value before the vehicle, and the lane area canthus be accurately extracted.

A third alternative of a road area extraction apparatus according to thesecond embodiment of the present invention is described next below withreference to FIGS. 37 to 39. This road area extraction apparatuscomprises a digital image pickup 41, brightness histogram generator 42,image separator 43, lane area clustering unit 44, and lane widthdetector 45.

The digital image pickup 41 is typically a video camera used forcapturing an image of the road in front of the vehicle as described inthe preceding alternatives.

The brightness histogram generator 42 produces a brightness histogramfrom the input road image data as shown in FIG. 38. Using thisbrightness histogram, the brightness histogram generator 42 also detectsthe vehicle position in the image, i.e., detects the luminance at thebottom middle part of the image.

The image separator 43 then detects all valleys in from the brightnessdistribution, producing a segmented lane image as shown in FIG. 39.

Based on the lane width information from the lane width detector 45, thelane area clustering unit 44 merges the segment below the vehicle withthe first horizontal segment, and then with a segment in the verticaldirection. The lane width detector 45 obtains the lane width from theimage height based on the lane area obtained from the lane areaclustering unit 44.

The road area extraction apparatus thus comprised operates as describedbelow with reference to the flow chart in FIGS. 40 and 41.

The first step is to generate a brightness histogram as shown in FIG. 38from the luminance data captured by the digital imaging pickup (stepS1).

Using the brightness b0 at the bottom center of the image, the positionin the histogram corresponding to the road surface directly below thevehicle is obtained (step S2). A valley on the left side is detectedfirst

Mean brightness values b_(mean) 1 and b_(mean) 2 are then obtained forthe overlapping brightness regions b0-b1≦b<b0 and b0-3b1/2≦b<b0-b1/2where b is a brightness parameter and b₁ is a constant brightness level(step S3).

The equation b_(mean) a=b_(mean) 2-b_(mean) 1 is then calculated (stepS4), and the sign of b_(mean) a is then determined (step S5). A valleyis detected by determining when the sign changes from negative topositive.

If the sign has not changed from negative to positive, b₀ =b₀ -b₁ /2(step S6), and the procedure loops back to step S3 (step S6). Thisprocess is repeated to the left side of the histogram (step S7), and theprocedure then advances to step S8.

A similar process is then executed to detect a valley on the right.

The first step (step S8) is to reinitialize b₀ to the value detected inS2, and then calculate b_(mean) 3 and b_(mean) 4 for the regions b₀<b≦b₀ +b₁ and b₀ +b₁ /2<b≦b₀ +3b₁ /2. The equation b_(mean) b=b_(mean)4-b_(mean) 3 is then obtained (step S9).

A valley is detected by determining when the sign changes from negativeto positive (step S10, FIG. 41) If the sign has not changed fromnegative to positive, b₀ =b₀ +b₁ /2 (step S11), and the procedure loopsback to step S7 (step S11). This process is repeated until a valley isdetected to the right side.

Whether valleys have been detected to the right side of the histogram isthen determined (step S12). If not, steps S11 and S8 are repeated. Ifstep S12 returns YES, the pixels in the brightness region between thedetected valleys are then detected as the lane area candidate region ofthe image (step S13). FIG. 39 represents the input image separated intothe areas between the detected valleys.

The area directly below the vehicle is then detected (step S14). RegionS0 in FIG. 39 is this target region. Target region S0 is then combinedwith the region S1 adjacent thereto (step S15).

The previously obtained lane width wf(h) is then referenced to calculatehorizontal width w(h) (step S16).

If wf(h)>w(h) (step S17), the procedure loops back to step S15. If wf≦w(step S17) and the height h of the combined regions is greater than adefined height h0, the procedure terminates.

Note that h0 is the height of the vanishing point of the road, which isa constant height determined by the installation height of the digitalimage pickup 41 if the road is level. If h<h0, the combined region iscombined with the region adjacent thereto above and narrower than thetop width of the lower combined region; in FIG. 39 regions S2 and S3 arecombined. The procedure finally terminates when the height h=h0.

By thus segmenting the image based on the lane width, and using knownproperties of the road configuration in the image to extract the lanearea, accurate lane area extraction is possible. Note that the lane areacan also be extracted by segmenting the image using a brightness patterncomparison in combination with the method described above.

A fourth alternative of a road area extraction apparatus according tothe second embodiment of the present invention is described next belowwith reference to FIGS. 42 to 44. This road area extraction apparatuscomprises a digital image pickup 51, brightness histogram generator 52,image separator 53, lane area clustering unit 54, lane width detector55, lane region correlation detector 56, and lane extending directiondetector 57.

The digital image pickup 51 is typically a video camera used forcapturing an image of the road in front of the vehicle as described inthe preceding alternatives.

The brightness histogram generator 52 produces a brightness histogramfrom the input road image data as shown in FIG. 38. Using thisbrightness histogram, the brightness histogram generator 52 also detectsthe vehicle position in the image, i.e., detects the luminance at thebottom middle part of the image.

The image separator 53 then detects all valleys in from the brightnessdistribution.

Based on the lane width information from the lane width detector 55, thelane area clustering unit 54 merges the segment below the vehicle withthe first horizontal segment, and then with a segment in the verticaldirection. The lane width detector 55 obtains the lane width from theimage height based on the lane area obtained from the lane areaclustering unit 54.

The lane region correlation detector 56 calculates the correlationbetween the lane area detected by the lane area clustering unit 54, anda triangle defined by a particular point on the vanishing line and twolane edge positions at the bottom of the image supplied from the lanewidth detector 55.

The lane extending direction detector 57 detects the direction of thelane relative to the vehicle from the point on the vanishing lineproviding the highest correlation value.

The road area extraction apparatus thus comprised operates as describedbelow with reference to the flow chart in FIG. 42.

The lane width is first detected (step Si) using the method of the thirdalternative of the second embodiment described above.

A point r on the vanishing line is then initialized to 0, i.e., to thecoordinate value at the left edge of the vanishing line (step S2).

The correlation between the extracted lane area and the triangle formedusing point r=0, i.e., the number of matching pixels, is thencalculated.

Point r is then incremented to r=r+1 (step S4), thus moving thevanishing point one pixel to the right. If r does not exceed pixel rh atthe right edge of the image (step S5), the procedure loops back to stepS3 to repeat the correlation calculation. If r is beyond the right edge,the value of r resulting in the highest correlation is detected (stepS6). If r is greater than half (rh/2) the number of horizontal pixels inthe image, the lane is to the right of the vehicle. If r<rh/2, the laneis to the left of the vehicle.

By thus obtaining the correlation between the detected lane area and atriangle formed between the vanishing point and the lane edges, thedirection of the lane relative to the vehicle can be easily obtainedusing the maximum correlation value.

A local positioning apparatus according to the first embodiment of theinvention can thus separately detect the contour points of the road areaand lane marker candidates using low and high frequency images with nospatial frequency overlap. The effects of noise can therefore be reducedwhen compared with lane extraction using only a high frequency image,the method of the prior art, and lane markers can be detected with highprecision.

The lane area can also be accurately detected in the road image by meansof the road area extraction apparatus according to the second embodimentof the invention by detecting the lane area from the image data, anddetecting a lane area polygon starting from the bottom center of theimage, a point corresponding to directly in front of the vehicle.

The lane area can also be accurately detected in the road image bydetecting the area containing the luminance values of the pixelsdirectly in front of the vehicle from a brightness histogram, and usingthe image area corresponding thereto to detect the edges of the lanearea starting from the bottom center of the image, a point correspondingto directly in front of the vehicle,

The lane area can also be accurately detected by detecting patternsidentical to the brightness distribution pattern directly before thevehicle.

The lane area can also be accurately detected by detecting the areacontaining the luminance values of the pixels directly in front of thevehicle from a brightness histogram, calculating the brightness valuescontaining the frequency of the number of pixels in the lane area of theimage, which is previously determined, and defining the pixels in thisarea as the lane area.

The lane area can also be accurately detected by calculating imagesegments by a road width and luminance pattern comparison, and usingknown features of the road configuration in the image to extract thelane area.

The invention being thus described, it will be obvious that the same maybe varied in many ways. Such variations are not to be regarded as adeparture from the spirit and scope of the invention, and all suchmodifications as would be obvious to one skilled in the art are intendedto be included within the scope of the following claims.

What is claimed is:
 1. A local positioning apparatus for use indetecting a local position of a subject, capable of advancing in adirection relating to a lane in a local area, based on a digital imagesignal representing a local image of an area in an advancing directionof the subject, said apparatus comprising:a first image signal generatoroperable to extract a high spatial frequency component from the digitalimage signal, and to produce an edge signal based on the high spatialfrequency component; a contour extractor operable to extract a contourof a lane based on the edge signal, and to generate road contour dataindicative of the contour of the lane; a second image signal generatoroperable to extract a low spatial frequency component from the digitalimage signal, and to produce a luminance signal based on the low spatialfrequency component; a lane area extractor operable to extract an areaof the lane based on the luminance signal, and to generate road regiondata indicative of the area of the lane; and a lane detector operable todetect a position of the lane based on the road contour data and theroad region data, and to produce a lane detection signal indicative ofthe position of the lane.
 2. A local positioning apparatus as claimed inclaim 1, whereinsaid lane area extractor comprises a lane region setteroperable to add a particular tolerance range to the road region data soas to produce second road region data representing a lane candidatearea, within edges defined by the tolerance range, in which a lane maybe present; and said lane detector is operable to compare the roadcontour data and the edges of the second road region data so as todetermine whether the contour of the lane is within the lane candidatearea.
 3. A local positioning apparatus as claimed in claim 2, whereinsaid lane detector defines the road contour data as the lane detectionsignal when the contour of the lane is within the lane candidate area.4. A local positioning apparatus as claimed in claim 2, wherein saidlane detector is operable to control said contour extractor to extract anew contour when the contour of the lane is not within the lanecandidate area.
 5. A local positioning apparatus as claimed in claim 1,further comprising an in-lane position detector operable to detect aposition of the subject in the lane based on the lane detection signal.6. A local positioning apparatus as claimed in claim 1, wherein saidlane area extractor is operable to extract pixels in the luminancesignal in a particular luminance range so as to produce the road regiondata.
 7. A local positioning apparatus as claimed in claim 6, whereinsaid lane area extractor comprises a first pixel luminance extractoroperable to extract a first luminance from a pixel representing afarthest point, which is a first distance from the subject, and toextract pixels with luminance below the first luminance to generate theroad region data.
 8. A local positioning apparatus as claimed in claim7, wherein said lane area extractor further comprises a second pixelluminance extractor operable to extract a second luminance from a pixelrepresenting a point closer than the first distance, and to extractpixels with luminance between the first luminance and the secondluminance so as to generate the road region data.
 9. A local positioningapparatus as claimed in claim 7, wherein the first distance is setaccording to the speed of travel of the subject.
 10. A local positioningapparatus as claimed in claim 7, wherein said first pixel luminanceextractor is operable to extract the first luminance from a pixelrepresenting an image at a particular horizontal position on ahorizontal line passing through the pixel representing the farthestpoint.
 11. A local positioning apparatus as claimed in claim 10, whereinsaid first pixel luminance extractor is operable to determine theparticular horizontal position ad hoc according to a steering angle ofthe subject.
 12. A local positioning method for use in detecting a localposition of a subject, capable of advancing in a direction relating to alane in a local area, based on a digital image signal representing alocal image of an area in an advancing direction of the subject, saidmethod comprising:extracting a high spatial frequency component from thedigital image signal, and producing an edge signal based on the highspatial frequency component; extracting a contour of a lane based on theedge signal, and generating road contour data indicative of the contourof the lane; extracting a low spatial frequency component from thedigital image signal, and producing a luminance signal based on the lowfrequency component; extracting an area of the lane based on theluminance signal, and generating road region data indicative of the areaof the lane; and detecting a position of the lane based on the roadcontour data and the road region data, and producing a lane detectionsignal indicative of the position of the lane.